In the field of imagery and image creation, the most time-consuming and error-prone aspects of algorithms used for generation of novel viewpoints from a plurality of images, is the correspondence process; namely, finding correct correspondences between the features of two or more images. Correspondences between features of two or more images are usually found by an automated technique that compares (or “matches”) areas of one image with areas of another image. It is very expensive for a correspondence process to compute a complete match of every feature in the image.
Most conventional techniques use per-pixel search in the image matching step of determining correspondences. Some selective techniques are often employed in order to reduce the amount of processing required in the image-matching step. Moreover, per-pixel search based matching (“PPSBM”) even with selective processing, causes video processing systems to perform poorly, both with respect to quality and time consumption, making real-time provision of virtual viewpoints very costly.
The selective techniques that limit processing time for PPSBM often employ temporal change detection. However, such change detection techniques detect changes in many areas that are part of the background. For example, the shadows of moving foreground objects that are usually projected onto background objects are detected. As a result, these photogrametric changes give rise to significantly more processing than is necessary. Furthermore, interior points of objects are not detected using conventional techniques, leading to errors in matching that subsequently lead to errors in the correspondences. Furthermore, PPSBM tends to give sparse results (few correspondences with respect to the number of pixels in the image), whereas dense results (up to the number of pixels in the image) are needed. Furthermore, PPSBM can give noisy results inside objects where contrast is low or features are lacking.
Thus, there is a need for a correspondence-finding method that creates dense correspondence fields, is less time consuming, and reduces processing errors without impairing match quality.